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NOTE 

This  is  a  revised  version  of  a  previous  research  report, 
"Extensions  of  a  Theorem  by  Clark,”  by  A.  Charnes,  W  W  Cooper , 
and  G  L  Thompson,  issued  as  ONE  Research  Memorandum  No  h2,  by 
the  Systems  Research  Group,  A  Charnes,  Director,  The  Technologies], 
l.istitute  and  Hie  Transportation  Center,  Northwestern  University, 
August  10,  1^61  The  present  report  see  Section  5ff> — extends  the 
previous  work  to  include  the  general  case  of  mixed  systems  of 
equations  and  inequalities,  which  had  not  been  included  in  the 
previous  release 
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I .  Introduction 

In  [3],  F.  £  Clark  established  the  following  results 
THEOREM  If  a  linear  programming  problem  [  written  in  inequality 
form]  has  a  solution,  then  either  the  primal  constraint  set  or  the 
dual  constraint  set  is  unbounded  The  parenthetical  remark  is  our 
addition  to  the  statement  of  the  theorem,  but  is  implicitly  assumed 
by  Clark,  as  any  reader  of  his  paper  will  be  aware  The  inequality 
form  of  the  general  linear  programming  problem  is* 

Min  wb 
subject  to 

wA  >  c 
w  >0 


(1) 


Max  cx 
subject  to 

Ax  <  b 
x  >  0 


This  is  sometimes  referred  to  as  the  canonical  form  and  used  as  a 

y 

standard  point  of  departure  far  further  analyses.,  let  some  care 
may  be  needed,  as  is  true  here,  if  a  full  degree  of  generality  and 
understanding  is  to  be  achieved  Note,  for  instance,  that  the 
parenthetical  remark  must  be  included,  as  above,  or  the  stated  theorem 
is  vulnerable  to  counter  examples  For  consider  the  following  self-dual. 


linear 

programming  problem; 

•- 

Max  1  ^ 

Min  JJ-  i 

(2  L) 

subject  to 

subject  to 

f  '  1 

7^-1 

y  Vide  rEj  pp  6h  ff  ,  from  which  we  quote,  as  follows;"  aueh 

problem  pairs  [of  mixed  equations  and  inequalities  ]  are  essentially 
no  more  general  than  the  canonical  ones  „  " 


Here  both  the  primal  and  the  dual  constraint  set  consist  of  exactly 
one  point  and  are  certainly  bounded 

It  is  true  that  the  latter  system  is  not  in  canonical  form  Also 
the  following  equivalent  system,  which  ia  in  canonical  form. 


Max  x^  :x? 


Min  w.,  - 


Subject  to 


Subject  to 


(2  2) 


xL  -  x2  <  1 


Vl  '  -  1 


-xi  •  x?  s  -1 


"W1  *  u2  - 


V  x?  -  0 


v?L,  v0  >  0 


obviously  has  both  constraint  seta  non-empty  end  unbounded  Evidently 
the  theorem  as  stated  does  not  provide  enough  light  to  illuminate  fully 
what  is  involved  even  for  the  two  simple  situations  displayed  in  (2  1) 
end  (2  2)  A  further  development  and  extension  therefore  appears  to  be 
in  order. 

While  it  is  true  that  every  linear  programming  problem  written 
in  inequality  form  can  be  written  as  an  equivalent  problem  in  equality 
form,  and  vice  versa ,  it  is  important  to  observe  that  this  concept  of 
equivalence  does  not  imply  that  the  constraint  sets  in  the  two  ways 
of  writing  the  problem  are  identical  or  even  have  similar  properties 
Examples  (21)  and  (2  2)  snow  that  boundedness  properties  of  equivalent 
problems  may  be  widely  different  All  that  the  concept  of  equivalence 

implies  is  that  from  a  solution  of  a  linear  programming  problem  in  one 
form  it  is  possible  to  obtain  a  solution  in  the  other  form 
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In  the  first  part  of  this  j;aper  we  reformulate  the  above  theorem 
to  obtain  a  sharper  and  more  general  statement  which  we  prove  via  a 
linear  programming  formulation  that  gives  direct  access  to  the  duality 
theorem  of  linear  programming  Thau  we  show  that  the  situation  of  (2.1) 
is  essentially  the  only  possible  exception  if  the  constraints  involve 
variables  that  are  unrestricted  as  to  sign  and  involve  only  equality 
constraints  We  show  that  any  such  problem  is  equivalent,  in  a  sense 
to  be  defined,  to  a  problem  having  these  characteristics  We  then 
return  to  a  further  variant  of  the  above  examples,  (2-1)  and  (2  2) 
and  then  establish  a  general  theorem  for  mixed  systems  of  equations  and 
inequalities  Finally,  we  consider  unconstrained  problems  that  are 
projection  equivalent  to  such  mixed  problems 

2  Proof  of  the  Dual  Constraint  Theorem 

Since  Clark  8  result  is  really  one  concerning  dual  sets  of 
constraints,  we  shell  reformulate  it  more  generally  as  follows* 

THEOREM  1  If  X  **  {x  /  Ax  <  b,  x  >  0  is  nonempty  and 
bounded  then  W  •»  lw  /  wA  >  c ,  w  >  0}  is  nonempty  and 
unbounded j  also  if  W  is  nonempty  and  bounded  then  X  is 
nonempty  and  unbounded 
PROOF  Consider  the  dual  problems 


Max 

ex 

Min 

wb 

subject  to 

subject 

to 

Ax  <  b 

wA 

X 

V 

o 

w 

>  0 
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where  e  «  (1,  1,  ,  ,  1}  If  X  is  nonempty  and  bounded 

then,  by  the  dual  theorem  of  linear  programming ,  there 
exist  optimal  solutions  x#  to  the  maximization  problem  and 
w*  to  the  maximization  problem  and  w#  to  the  minimization 
problem  Of  course,  such  a  w#  satisfies  w#  >  0  and  w*A  >  e 
Now  let  a  -  max  (1,  c. ,  ,  c  *  for  any  collection 

c  "  (c^ ,  „ ■ ,  c^).  Then  for  all  a  >  defining  w  »  aw# 

we  have 

wA  *  a  w#A  >  a  e  >  a  e  >c. 

—  -  o  - 

Also,  since  w*  >  0  it  is  clear  that  w  >  0  Thus  W  contains 
the  infinite  ray 

{w  !  v  «  aw#,  a  >  a  } 

*  o 

and  is  thereby  unbounded 

On  the  other  hand,  if  W  «  <w  /  wA  >  c,  w  >  0)  is  nonempty 
and  bounded,  we  observe  that  equivalent  expressions  for  X  and 
W  are  • 

W  -  tw  /  (  A ) TwT  <  .cT,  wT  >  0} 

X  «  lx  /xT(- A)T  >  (  -b)T,  xT  >  0), 


where  T  is  the  transpose  operation.  Then,  to  complete  the 
proof,  we  need  only  consider  the  dual,  linear  programming 
problems 


Max  (  b)TwT 


Min  xT(  *)T 


Subject  to 

t  -A)  V  <  (  c) 


.nU  .  I  -T 


T 

wl  »  0 


Subject  to 

XT(./0T  ;  <  b)T 

xT.o  . 
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and  apply  the  previous  result  to  show  that  X  is  nonempty  and 
unbounded 

3 -  Dual  Problems  in  Unrestricted  Variables 

Next  we  consider  the  dual  problems  in  unrestricted  variables. 

Max  c  £  Min  TJ  b 

(3)  subject  to  subject  to 

A^-b  7]  A  ■«  c , 

in  order  to  prove  the  following  theorem 

THEOREM  2  If  ^  £  /  A  £  *■  b)  is  nonempty  and  bounded 

and  H  -  <7f/77*  -  cl  ia  bounded,  then  both  I— 1  and  |“j 
consist  of  a  single  point., 

PROOF  Suppose  that  there  could  be  two  points,  ^  /  £ 

in  -  Then  £  ^  satisfies  A(  £,-  £?>-o 

Hence  the  infinite  lire 

f  £  I  £  *  £  l  *  a(>  £  l  S  aU  real  al 

is  contained  in  ,  contradicting  the  boundedness  which  was 
assumed  to  hold  for  this  solution  set  Therefore,  if  is 

nonempty  and  bounded  it  must  consist  of  only  one  point 

By  the  dual  theorem,  we  Immediately  have  H  is  nonempty 
But  then  if  r  “I  is  also  bounded,  8n  argument  which  is  wholly 
analogous  to  the  one  already  given  shows  that  H  cannot 
consist  of  more  than  one  point 

Another  result  concerning  linear  programming  problems  in 
unrestricted  variables  is  the  following 


6- 


THEOIIEM  3  If  ia  nonempty  end  bounded  then  and 

H  each  consist  of  only  one  point  when  the  rows  of  A  are 
linearly  independent,  end  (b)  consists  of  one  point  and 
H  is  unbounded  when  the  rows  of  A  are  linearly  dependent 
PROOF  For  case  a)  we  have,  by  the  previous  argument,  that 
consists  of  only  one  point  and  hi  is  nonempty  Suppose 
that  hi  could  contain  two  distinct  points  Then 

j  2  ^  0  But  (  7^  ^  _  7|  ^)A  *  0;  i.e  ,  the  rows  of 

A  are  linearly  dependent  -e  contradiction  Thus  H  consists 
of  only  one  point 

For  (b)  wo  obtain  that  -S.  consists  of  a  single  ^oint  and 
H  nonempty  as  before  Since  the  rows  of  A  are  linearly 


dapen  ent,  there  must  exist  an  7|  /  0  such  that  7^  A  «■  0 
If  7^  is  any  point  in  H  then  77  a  is  also  inH  for 
all  real  a  This  means H contains  an  infinite  lino  end  hence 
it  is  unbounded 


h  F rojeetlon  Equivalence 

It  is  very  easy  to  change  a  pair  of  constraint  sets  of  the  type 
described  in  Theorem  2  (of  which  (2.1)  is  a  specific  example)  into  a 
pair  of  constraint,  seta  both  of  which  are  unbounded  This  can  be  done 
by  merely  adding  a  row  and  a  column  of  zeros  to  A  and  adding  a  zero 
entry  to  each  of  the  vectors  b  and  c  This  operation  turns  the 
constraint  sets  into  "cylinder  sets"  in  one  higher  dimension,  and  such 
seta  are  always  unbounded 


7  ■ 


The  converse  operation  is  also  possible,  namely,  if  A  has  a 
row  or  a  column  that  is  all  zero  and  if  the  corresponding  entry  in 
b  or  c  is  also  zero,  then  it  is  possible  to  project  the  problem  into 
a  lower  dimension  by  eliminating  the  corresponding  direct  or  dual 
variable 

We  elaborate  on  this  idea  as  follows:  first  recall  (soe  fl] 
p  '/60}  Theorem  13)  that  if  A  is  an  mxn  matrix  then  there  exist 
nonsingular  matrices  S  and  T  so  that 


a  i) 


SAT  -  D 

where  D  is  a  matrix  of  the  form 


I  0  ,  , 

r  x  r  rx( n  r) 


°(m-r)xr  °(  i  r)x(n-r)< 


with  I  representing  the  Identity  matrix,  0  a  zero  matrix  and  the 
subscripts  are  used  to  indicate  the  dimensions  in  each  case  Evidently 
the  ra.ik  of  A  is  r  Hence,  if  we  define  the  projection  matrices  P  end 
Q  as  follows 


p  .  t]  n  ) 

rxm  •  rxr  rx(rc  r) 


then  we  see  that 


nxr 


rxr 


'(n-r) 


xr' 


a  3) 


PSATQ  »  PDQ  -  I 

rxr 


Here  p  projects  m  dimens .onal  column  vectors  into  r  dimensional, 
row  vectors,  while  Q  projects  n  dimensional  row  vectors  into  r 
dimensional  row  vectors 
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We  now  return  to  the  dual,  problems  (3)*  and  introduce  changes 
in  variables  of  the  form 

(5)  £  -  TO* 

(6)  -  yPS 

where  P  and  Q  are  projection  matrices  as  above  (except  that  r 
herfc  need  not  be  the  rank  of  A),  S  and  T  are  nonsingulars  and  T 
is  such  that  the  last  nr  columns  of  AT  are  all  zeros  end  S  is 
.such  that  the  last  ru-r  rows  of  SA  are  all  zeros  It  is  clear  inat 
r  is  greater  then  or  equal  to  the  rank  of  A. 

V\e  now  want  to  introduce  the  idea  of  projection  equivalent  linea 
programming  problems  and  therefore  produce  the  following  dual  linear 
programme ng  problems ; 

Max  cTQjc  Min  yPSb 

subject  tc  subject  to 

(7) 

PSATQ*  ~  PSb  yPSATQ  -  cTQ 

Then  we  say  that  the  linear  programming  problems  (3)  and  ( 7 )  are 
projection  equivalent  when  we  can  obtain  a  solution  to  one  of  them 
from  the  solution  to  the  other  by  means  of  the  change  of  variable 
equations  (5)  and  (6'* 

THi’O.lEM  h  If  the  linear  programing  problem  in  unrestricted 
variables  (3,  has  a  solution  then  it  is  projection  equivalent 
to  tne  problem* 

Max  uTQx  *  Min  yPSb 

* 

ra\  subject,  to  subject  to 


x  -  PSb 


y  ~  cTQ 


9- 

where  P,  S,  A,  T,  and  Q  satisfy'  (h  3)  and  r  ie  the  rank 
of  A, 

Remark  Note  that  the  dual  constraint  sets  of  (8) 
caeh  consist  of  a  single  point  and  hence  are  bounded 
PR001  As  already  remarked,  given  A,  it  is  possible  to 
find  P,  5,  T,  and  Q  so  that  (U  3)  is  satisfied  Hence 
the  changes  of  variables  (5)  and  (6)  ore  well  defined,  with 
r  being  the  rank  of  A  The  solution  to  (6)  is  obvious  and 
the  two  dual  problems  share  the  common  value  cTQPSb  If  we 
set 

£  *  TOPSb  and  7^  -  cTQPS 

then  these  values  will  make  the  objective  functions  of 

(3)  shaie  a  common  value  and  this  will  be  an  optimum  for 

(3)  if  these  £  and  7^  satisfy  the  constraints  To  show 

this  we  observe  first  of  all  that  A  -  S  TIT  1  and  that 
T 

QP  **  D  Then  we  have 

(S')  A  £  *  ATQPSb  -  S  LDT  1TDTSb  -  S^Dl/Sb  -  b 

The  final  reduction,  indicated  on  the  right,  follows 
from  the  assumption  that  (3)  has  a  solution  For,  from 
SA  *  DT  \  and  the  definition  of  D  -  see  (It  1)  -  we  observe 

that  the  last  m  r  rows  of  SA  consist  entirely  of  zeros 
This  implies,  if  (3)  has  a  solution,  that  the  last  m  r 
entries  of  Sb  consists  entirely  of  zeros;,  for  otherwise  the 
equations  A  £  «  b,  which  are  equivalent  to  SA  £  -  Sb,  would 


be  inconsistent 


Thus  on  the  assumption  that  (3)  has  a  solution  we  have 
established  £  -  TQPSb  satisfies  the  constraints  of  this 
problem  Similarly,  one  can  show  that  m  cTQPS  satisfies 
the  dual  constraints  and  therefore  these  values  for  £  and 
7^  are  optimal  for  (3) 


$  The  Mixed  Constraint  Case 

As  we  observed  immediately  after  (2  2),  the  geometric 
properties  ‘o'  equivalent  linear  programming  problems  may  be 
strikingly  different  In  fact,  quite  trivial  alterations  can 
bring  this  about  for  instance,  one  can  always  replace  a  problem 
in  which  the  rows  of  A  are  linearly  independent,  by  an  equivalent 
problem  whose  irwtrix  has  linearly  dependent  rows--e  t  ,  by  merely 
repeating  some  of  the  originally  stated  constraints --in  order  to 
bring  Theorem  3  into  play  and  alter  the  dual  solution  set  from  one 
which  is  bounded  to  one  which  is  unbounded 

Preparatory  to  dealing  with  the  general  case  of  mixed  equations 
and  inequalities,  we  exhibit  still  another  variation  of  (21)  and 
(2  2)  Evidently  the  problem 


(10  L) 


Max  £ 
Subject  to 


£  - 1 
£? 0 


doos  not  alter  any  solution  property  for  the  direct  problem  in  (?.  I'1 
The  only  point  in  HI.  continues  to  be  £  1  The  dual  to  (10  1),  viz 


ii- 


do  2) 


Min  7j? 
Subject  to 


7?>3 

admits  of  an  unbounded  solution  set,  H  a  so  that  the  more  adjunction 
of  a  redundant  non-negativity  requirement  produces  e  markedly 
different  situation  from  the  dual  to  (2  1) 

It  is  this  kind  of  situation  that  Clerk's  theorem  fails  to 
illuminate  and  so  we  now  elaborate  for  the  general  situation  as 
follows 


THEOREM  5  If 


55-  is  nonempty  and  bounded  for 

M,x  ci  £  i  *  £  a 

Subject  to 


the 


n  H 


A  P 

11  S  1 

•  \ ?  £ 

’?  -  bl 

A  ,  £. 

*  A  £ 

<  b 

21  S  1 

2?  S 

1 

2  -  2 

unbounded  for 

Min  bL 

*  7? 

(  2 

b2 

Subject  to 

Vu  * 

^/21 

*  C1 

Vi?  • 

^1/  22 

>  C2 

T* :  0 


PROOF 


If  ™  is  nonempty  and  bounded  then  a  finite  optimum 


exists  for  every  p  in  the  problem 


1? 


Max  (ci  f  i *  v- e? 

Subject  to 


Ani 

A21  i 

r  i  *  *??  s b? 

f2:° 

where 

p  is  a  scalar, 

A^j ,  for  i ,  j  1 ,  < 

of  the 

matrix  A,  and 

is,  as  before,  a 

components  equal  to  unity.  By  the  duality  theorem,  the 

following  problem  then  also  has  a  solution  for  every  p 

* 

Min  7Jlbl  -  ^ 


Subject  to 

*  ‘  l 


'Via  * 

V?2  ?  “  °2 

»f,:  » 

Since  Ke  .  >  c 

C  i 

for  some  K 

>  0,  every  solution  with  p  ^  K 

is,  a  fortiori, 

a  solution 

of 

'1 2*2).  '  °1 

'lit ,  ’ 

7I-/?? ;  °2 

• 

1  1 

end  hence  es  in  H 


Suppose,  ontrary  to  the  assertion  of  the  theorem,  that 
H  were  bounded  But  H  bounded  implies  ther^  exist 


- 16 


K  ^  >  Oj  i  *  1,  2,  such  that  -  K  <  7 ^  <  K 

for  all  (  71  ^  ?|  ' ,  H  And  also 


Vi2  * 


Me 


for  some  M  >  0  This,  however,  contradicts  the  assumed 
existence  of  a  solution  satisfying 


Vl,  1  ^  2*2?  ; 2  M'; 


Hence  H  must  oe  unbounded 


Q  E„  D 


Our  final  resuJt  shows  that  it  is  possible  to  reformulate  any 
linear  programming  problem  that  has  a  solution  in  b  finite  number  of 
ways  so  that  both  its  constraint  sets  are  bounded  one  point  sets 

THEOREM  6  Any  solvable  linear  programming  problem  is 

projection  equivalent  to  a  finite  number  of  linear  programming 
problems  having  one -point  (hence  bounded)  dual  constraint 
sets 

PROQi  Suppose  that  the  dual  constraints  of  the  problem  «re 
A  £  <  b  and  7^  A  >  c,  where,  as  in  the  statement  of  Theorem 
some  of  the  constraints  are  equalities  and  some  inequalities, 
and  the  nonnegativity  conditions  are  correspondingly  determined 
let  0  and  be  solutions  to  the  linear  programming  problem 
Then  these  two  vectors  satisfy  the  above  constraints,  together 
with  any  nonnegati vity  constraints,  and  also  satisfy  7^°b  *  0 


But  then 


t\°  °  <  -  c£°<  7J°a£° 


ih- 


This  moans  that  whenever  one  of  the  constraints  is  satisfied 

as  a  strict  inequality  the  corresponding  dual  variable  is 

*  •  * 


necessarily  zero  Now  let  A  ,  b  ,  and  c  be  the  corresponding 
matrices  with  rows  or  columns  deleted  corresponding  to  zero 
components  of  £°  and  7J°  Let  £ 0  and  7J°  be  similarly 

defined  Then  by  the  above  remarks  we  have 

A'  £  °  *  b>  >  A  -  c',  and  T}°  b  -  c'f° 

and  also  any  remaining  nonnegativity  conditions  are  still 
satisfied  Hence  f'  and  7?°  are  solutions  to  the 
unconstrained  dual  problems 

Max  c'f  .  Min  7J  b' 

Subje  t  to  Subject  to 

S  **  t>  7^a*c 


Moreover,  among  the  solutions  to  the  latter  unconstrained 
problem  will  be  found  ones  which  satisfy  the  desired  nonnegativity 
eond„tions  that  may  remain,  and  hence  will  be  solutions  to  the 
original  oblem  Hi  us  the  unconstrained  problem  is  projection 

equivalent  to  the  original  problem,  in  the  sense  that  among  the 
solutions  to  each  problem  appear  solutions  to  the  othor 

be  can  now  apply  Theorem  li  and  make  further  projection 
equivalence  of  the  unconstrainsd  problem  just  found  with  one 
whose  dual  constraint  sets  are  one -point 

Finally,  we  remark  that  although  there  can  be  infinitely 
many  solutions  to  a  linear  programming  problem  it  is  evident 
thot  them  are  only  a  finite  number  of  ways  of  crossing  out 
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rows  and  columns,  hence  the  above  described  process  will  lead 
to  only  a  finite  number  of  different  projection  equivalent 
problems 

6  Conclusion 

Via  (3)  and  (8)  and  their  associated  theorems,  we  have 
attempted  to  illuminate  one  aspect  of  the  geometric  properties 
associated  with  dual  pairs  of  linear  programming  problems  These 
results,  together  with  Theorems  5  and  6,  make  It  possible  either  to 
remove  or  to  adjoin  various  constraining  conditions  in  order  to 
obtain  suitable  geometric  properties  for  the  sets  ~~~*  and  H 
In  usny  cases,  it  is  desirable  to  secure  proofs  of  venous  theorems 

y 

by  constructing  so  called  linear  programming  chains  which  alter 
the  geometric  properties  of  direct  or  duel  solution  sots.  The  above 
theorems  can  bo  used  for  guidance  in  securing  the  geometric  properties 
that  nre  wanted  in  these  chains  of  ''equivalent"  problems 

There  are  obvious  connections  between  the  ''extended  theorem  of 
the  '.to motive (see  (?],  pp  5*>0  ff  end  p.&D  and  the  first  projection 
empioyed  in  the  proof  of  Theorem  6  This,  in  turn,  means  that  a  variety 
of  interesting  applications  (e  g  in  economics!  can  be  made  from  this 
-and  the  related  developments  in  this  paper  we  cannot  pursue  this  topic 
hero,  however  Instead,  we  conclude  by  observing  that  no  real  trouble 
eme  gea  from  these  developments  as  far  as  computations  or  solutions  are 
concerned  K very  constructive  method  for  solving  linear  programming 


1/  oee  \v] 
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problems  is  now  designed  so  that  it  does  not  require  one  to  examine 

more  than  a  bounded  set  Indeed,  because  of  so~cailed  regularization 

1/ 

procedures  that  have  now  been  developed,  the  computation  may  be 
prosecuted  in  a  routine  fashion  to  obtain  all  relevant  information 
even  when  a  problem  is  not  solvable  or  hi  s  an  infinite  optimum^  fails 
to  have  full  rank,  and  30  on 


« 


1/  Vide,  c  g  ,  Chapter  VI  ^na  Vli  in  [2]- 
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